Limiting video surveillance collection to authorized uses

ABSTRACT

Provided herein are systems and methods for automatically limiting video surveillance collection to authorized uses and authorized users. To achieve this control, the authorization system can be configured to manage and secure a plurality of crypto keys associated with encrypting a plurality of corresponding video footages and release a crypto key for a video footage at approved times to limit user access to the video footage. In particular, the surveillance system can generate a video collection including a copy of portions of a received video footage that include one or more approved events from a watchlist of a user. Thereafter, the video footage can be encrypted by a first key managed by the authorization system and prevents the user from accessing video content of the video footage once encrypted. Accordingly, the user may be limited to accessing the video collection and not all the portions in the encrypted video footage.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims the priority of U.S. Provisional Application No.63/077,417, filed Sep. 11, 2020, the entire contents of which areincorporated herein by reference.

FIELD

The present disclosure relates generally to collecting videosurveillance data and in particular, limiting collected videosurveillance data to authorized uses.

BACKGROUND

Video surveillance was initially performed using closed circuittelevision (CCTV) and was first installed in Germany in 1942 formilitary purposes to observe launch of V-2 rockets during theirdevelopment. Since then, video surveillance has become omnipresent andhas extended to a variety of non-military applications including inbanks, patient monitoring at hospitals, industrial machine monitoring,retail stores of all kinds, public and private venues with substantialtraffic volumes, and increasingly in private residences. For example,video monitoring has long been used to improve security through trafficmonitoring, crime reduction and deterrence, retail theft prevention,false liability claims prevention, loitering detection, prisonermonitoring, vandalism deterrence, etc.

Due to the technological improvements in digital cameras and memorystorage, large volumes of high quality pictures and video can beobtained from digital cameras and stored at relatively low costs. As aresult, in the past few decades, video footage has been over-collectedwhich not only prevents efficient review of collected video, but alsoraises privacy concerns and encroaches on the rights of individualscaptured in recorded video. Moreover, due to the higher qualitypictures, an ever increasing amount of data storage is required to storevideo content of the same duration. For example, during the 2005 Londonsubway bombing incident, over 6,000 hours of CCTV footage was stored.Even using computer-assisted review of the video footage has becomeincreasing resource intensive and inefficient especially consideringthat the 6,000 hours of video translated into about 22 terabytes (TB) ofdata.

Accordingly, it is desirable to limit video footage collection only toauthorized uses.

SUMMARY

As discussed above, current video surveillance systems over-collectvideo content and captures video of both events of interests and manybystanders, which raises significant privacy concerns. One approach toaddressing these concerns may include discarding portions of videofootage such that only approved events are captured in retained (i.e.,non-discarded) portions of the video footage. Regardless of thesurveillance application such as for private, commercial, or governmentuser, there is sometimes a need to access the original video footage.Therefore, there exists a need for systems and methods for controllingcompliance of video surveillance to authorized uses, which may changeover time.

Disclosed here in are methods for limiting video surveillance toauthorized uses, comprising: at a surveillance system: receiving a videofootage of an area under surveillance from a security camera; analyzingthe video footage to detect a plurality of events in the video footage;generating a video collection comprising a copy of a plurality ofportions of the video footage based on applying a watchlist indicatingan approved event of interest to the analyzed video footage, whereineach copied portion is detected to include the approved event;encrypting the video footage using a first key to restrict a first userfrom accessing other portions of the video footage that do not includethe approved event; and providing the first user access to the videocollection.

Some examples of the disclosure are directed to A surveillance systemfor limiting video surveillance to authorized uses, comprising: one ormore processors; memory comprising a local storage; and one or moreprograms, wherein the one or more programs are stored in the memory andconfigured to be executed by the one or more processors, the one or moreprograms including instructions for: receiving a video footage of anarea under surveillance from a security camera; analyzing the videofootage to detect a plurality of events in the video footage; generatinga video collection comprising a copy of a plurality of portions of thevideo footage based on applying a watchlist indicating an approved eventof interest to the analyzed video footage, wherein each copied portionis detected to include the approved event; encrypting the video footageusing a first key to restrict a first user from accessing other portionsof the video footage that do not include the approved event; andproviding the first user access to the video collection.

Some examples of the disclosure are directed to a non-transitorycomputer-readable storage medium comprising one or more programs fortargeted video surveillance processing, wherein the one or moreprograms, when executed by one or more processors, cause the one or moreprocessors to: receiving a video footage of an area under surveillancefrom a security camera; analyzing the video footage to detect aplurality of events in the video footage; generating a video collectioncomprising a copy of a plurality of portions of the video footage basedon applying a watchlist indicating an approved event of interest to theanalyzed video footage, wherein each copied portion is detected toinclude the approved event; encrypting the video footage using a firstkey to restrict a first user from accessing other portions of the videofootage that do not include the approved event; and providing the firstuser access to the video collection.

Additionally or alternatively to one or more examples disclosed above,the approved event comprises a class of targets or a specific target.Additionally or alternatively to one or more examples disclosed above,the watchlist comprises an indication of the approved event and aplurality of associated parameters for video surveillance, and themethod comprises: determining which video segments of the video footagemeets the plurality of parameters and includes the approved event,wherein the plurality of copied portions of the video footage comprisesthe determined video segments. Additionally or alternatively to one ormore examples disclosed above, the plurality of parameters comprises oneor more locations and one or more time periods approved for surveillingthe target. Additionally or alternatively to one or more examplesdisclosed above, the plurality of parameters includes a locationapproved for surveilling the event, and the method comprises:determining that the video footage is associated with the approvedlocation before generating the video collection. Additionally oralternatively to one or more examples disclosed above, determining thatthe video footage is associated with the approved location comprises oneof: analyzing the video footage to identify one or more features thatuniquely identifies the location; or determining that metadata of thereceived video footage indicates the location. Additionally oralternatively to one or more examples disclosed above, the methodcomprises: transmitting the first key to an authorization system thatcontrols access to a plurality of encrypted video footages; receiving anotification from the authorization system indicating that the first keywas received and securely stored by the authorization system; andirrecoverably delete the first key from the surveillance system.Additionally or alternatively to one or more examples disclosed above,the first key is a first public key, comprising: receiving the firstpublic key from an authorization system that controls access to aplurality of encrypted video footage, wherein the authorization systemstores a first key pair comprising the first public key and a firstprivate key, and wherein the encrypted video footage can only bedecrypted using the first private key located at the authorizationsystem. Additionally or alternatively to one or more examples disclosedabove, providing the users access to the video collection comprises:encrypting the video collection using a second key pair associated withthe user to prevent other users from being able to access the videocollection, wherein the video collection is decrypted using the secondkey pair when access to video content in the video collection isrequested by the user. Additionally or alternatively to one or moreexamples disclosed above, analyzing the video footage comprises:generating a metadata file comprising an index that maps each event fromthe plurality of targets to one or more portions of the video footagedetected to include the event. Additionally or alternatively to one ormore examples disclosed above, wherein the method further comprises:encrypting the metadata file with the first key. Additionally oralternatively to one or more examples disclosed above, generating thevideo collection comprises: identifying the plurality of portions of thevideo footage that include the approved event by searching the metadatafile; and copying the plurality of identified portions to the videocollection. Additionally or alternatively to one or more examplesdisclosed above, the method comprises: generating a second metadata filethat includes the index for the approved event and that removes orredacts the index to non-approved events of the plurality of events.Additionally or alternatively to one or more examples disclosed above,the approved event is a first approved event, wherein the watchlistindicates a second disapproved event whose captured video cannot beaccessed by the users, and wherein each copied portion is detected toinclude the first approved event and not include the second disapprovedevent. Additionally or alternatively to one or more examples disclosedabove, the method comprises receiving a request from an authorizationsystem to apply an updated watchlist associated with the users to theencrypted video footage; decrypting the encrypted video footageencrypted by a first encryption key using a decryption key received fromthe authorization system; generate a second video collection including acopy of a second plurality of portions from the decrypted video footagebased on the updated watchlist, wherein each portion comprises one ormore approved events from the updated watchlist; encrypting thedecrypted video footage using a second encryption key to restrict theusers from accessing other portions of the video footage not approved bythe updated watchlist; and provide the users access to the second videocollection.

BRIEF DESCRIPTION OF THE DRAWINGS

The present disclosure will now be described, by way of example only,with reference to the accompanying drawings, in which:

FIG. 1A illustrates a system including an authorization system forlimiting video surveillance executed by a surveillance system toauthorized uses, according to some embodiment;

FIG. 1B illustrates a system including an authorization system forlimiting video surveillance executed by a surveillance system toauthorized uses, according to some embodiments;

FIG. 2 illustrates a block diagram showing examples for how a videofootage is processed to generate video collections accessible byrespective users, according to some embodiments;

FIG. 3 illustrates a block diagram showing an example for how an updatedwatchlist of a user can be applied to a previously captured videofootage, according to some embodiments;

FIG. 4 illustrates a block diagram of a system for authorizing andexecuting video surveillance, according to some embodiments;

FIG. 5 illustrates a flowchart of a method for limiting videosurveillance to authorized uses, according to some embodiments;

FIG. 6 illustrates a flowchart of a method for authorizing videosurveillance on previously collected video footage controlling,according to some embodiments; and

FIG. 7 illustrates an example of a computer, according to someembodiments.

DETAILED DESCRIPTION

As discussed above, larger and larger volumes of video footage are beingcaptured and stored. Moreover, many bystanders or non-authorizedcollection targets are captured in the video. Therefore, there exists aneed for a video authorization system that controls how a videosurveillance system collects and stores video content such that onlyapproved targets are captured and whose video can be reviewed byapproved users. In some embodiments, to achieve this control, theauthorization system can be configured to manage and secure a pluralityof crypto keys associated with encrypting a plurality of correspondingvideo footages and release a crypto key for a video footage at approvedtimes to limit user access to the video footage. In particular, thesurveillance system can generate a video collection including a copy ofportions of a received video footage that include one or more approvedevents from a watchlist of a user. Thereafter, the video footage can beencrypted by a first key managed by the authorization system andprevents the user from accessing video content of the video footage onceencrypted. Accordingly, the user may be limited to accessing the videocollection and not all the portions in the encrypted video footage.

In the following description of the various embodiments, reference ismade to the accompanying drawings, in which are shown, by way ofillustration, specific embodiments that can be practiced. Thedescription is presented to enable one of ordinary skill in the art tomake and use the invention and is provided in the context of a patentapplication and its requirements. Various modifications to the describedembodiments will be readily apparent to those persons skilled in the artand the generic principles herein may be applied to other embodiments.Thus, the present invention is not intended to be limited to theembodiment shown but is to be accorded the widest scope consistent withthe principles and features described herein.

As used herein, the singular forms “a,” “an,” and “the” used in thefollowing description are intended to include the plural forms as wellunless the context clearly indicates otherwise. It is to be understoodthat the term “and/or” as used herein refers to and encompasses any andall possible combinations of one or more of the associated listed items.It is further to be understood that the terms “includes,” “including,”“comprises,” and/or “comprising,” when used herein, specify the presenceof stated features, integers, steps, operations, elements, components,and/or units but do not preclude the presence or addition of one or moreother features, integers, steps, operations, elements, components,units, and/or groups thereof.

Certain aspects of the present invention include process steps andinstructions described herein in the form of a method. It should benoted that the process steps and instructions of the present inventioncould be embodied in software, firmware, or hardware, and, when embodiedin software, they could be downloaded to reside on, and be operatedfrom, different platforms used by a variety of operating systems. Unlessspecifically stated otherwise as apparent from the following discussion,it is appreciated that, throughout the description, discussionsutilizing terms such as “processing,” “computing,” “calculating,”“determining,” “displaying,” or the like refer to the action andprocesses of a computer system, or similar electronic computing device,that manipulates and transforms data represented as physical(electronic) quantities within the computer system memories or registersor other such information storage, transmission, or display devices.

The present disclosure in some embodiments also relates to a device forperforming the operations herein. This device may be speciallyconstructed for the required purposes, or it may comprise a generalpurpose computer selectively activated or reconfigured by a computerprogram stored in the computer. Such a computer program may be stored ina non-transitory, computer readable storage medium, such as, but notlimited to, any type of disk, including floppy disks, USB flash drives,external hard drives, optical disks, CD-ROMs, magnetic-optical disks,read-only memories (ROMs), random access memories (RAMs), EPROMs,EEPROMs, magnetic or optical cards, application specific integratedcircuits (ASICs), or any type of media suitable for storing electronicinstructions, and each coupled to a computer system bus. Furthermore,the computers referred to in the specification may include a singleprocessor or may be architectures employing multiple processor designsfor increased computing capability.

The methods, devices, and systems described herein are not inherentlyrelated to any particular computer or other apparatus. Variousgeneral-purpose systems may also be used with programs in accordancewith the teachings herein, or it may prove convenient to construct amore specialized apparatus to perform the required method steps. Therequired structure for a variety of these systems will appear from thedescription below. In addition, the present invention is not describedwith reference to any particular programming language. It will beappreciated that a variety of programming languages may be used toimplement the teachings of the present invention as described herein.

FIG. 1A illustrates a system 100A including an authorization system 150for limiting video surveillance executed by a surveillance system 101 toauthorized uses, according to some embodiments.

In some embodiments, authorization system 150 can be managed by adifferent entity than surveillance system 101 and cannot be accesseddirectly by users of surveillance system 101. In some embodiments,authorization system 150 can be configured to manage watchlists 152 ofone or more events of interest (e.g., one or more persons or a specificperson) and their respective processing policies, as will be furtherdescribed below. For example, a watchlist may indicate an event andassign a collection policy that specifies parameters under which videocontent of that event can be collected. In some embodiments,authorization system 150 can be configured to be accessed by anauthorized user. Authorization system 150 may permit the authorized userto configure one or more watchlists 152 such as add or remove targets aswell as add, revise, or remove one or more processing policiesassociated with each target. In some embodiments, this authorizationuser possesses different credentials or a different role than asurveillance user that accesses surveillance system 101.

For example, in a law enforcement surveillance scenario, thesurveillance user may be an officer or other law enforcement personneland the authorized user may be associated with a court or a judicialoffice. Therefore, since the authorized user, e.g., a court official,controls which video content can be collected by the surveillance user,the surveillance user may be restricted to accessing captured video ofonly those events (e.g., persons of interest) previously approved by theauthorization user. In some embodiments, the surveillance user cansubmit a request to the authorization user or authorization system 150directly for an updated watchlist. For example, the surveillance usermay have subsequently received approval (e.g., via a granted warrant) tomonitor additional events. In this case, once authorization system 150verifies the request, authorization system 150 can update one or morewatchlists 152 and permit video managing system 120 to access additionalvideo content, as will be further described below.

In some embodiments, authorization system 150 can be configured tomanage and secure a plurality of crypto keys for securing a plurality ofrespective video footages generated by surveillance system 101. Forexample, authorization system 150 may store an association between oneor more keys 154 with a video ID 156 uniquely identifying a videofootage or a video feed (e.g., from a security camera) providing thevideo footage. As will be further described below, one or more keys 154may be used by surveillance system 101 to encrypt video footage suchthat the surveillance user cannot directly access video content in theencrypted video footage without express authorization from authorizationsystem 150.

In some embodiments, surveillance system 150 can implement one or moresymmetric-key encryption algorithms to encrypt the video footage. Inthese embodiments, key 154 may be a crypto key that can both encrypt anddecrypt video data. In some embodiments, when the surveillance user hasbeen approved to access encrypted video footage 132, authorizationsystem 150 can be configured to transmit, for example, key 154 to videomanaging system 120 to permit encrypted video footage 132 to bedecrypted. In some embodiments, as will be further described below,surveillance system 150 can be configured to delete a received key 154after decrypting and/or re-encrypting the video footage.

In some embodiments, surveillance system 150 can implement one or moreasymmetric-key encryption algorithms to encrypt the video footage. Inthese embodiments, key 154 may include a key pair of a public key and aprivate key. In some embodiments, video data that is encrypted by thepublic key can only be decrypted using the private key and vice versa.In these embodiments, when the surveillance user has been approved toaccess encrypted video footage 132, authorization system 150 can beconfigured to transmit, for example, a private key from a first key pairand a public key from a second key pair to video managing system 120.The private key may be used by surveillance system 150 to decryptencrypted video footage 132. After surveillance system 150 completesprocessing a decrypted video footage, surveillance system 150 can applythe public key of the second key pair to re-encrypt the video footage.Accordingly, the surveillance user can no longer access the re-encryptedvideo footage because the surveillance user does not possess a privatekey part of the second key pair secured on authorization system 150.

In some embodiments, surveillance system 101 includes a video managingsystem 120 for managing targeted video surveillance processing of videofootage from a plurality of surveillance areas 104A-C, according to someembodiments. System 100A includes a plurality of security cameras 106A-Dfor observing surveillance areas 104A-C within a field of view ofrespective security cameras 106A-D. For example, security cameras 106A-Bmay be installed at specific locations to monitor surveillance area104A. Likewise, security cameras 106C and 106D may be installed atspecific locations to monitor surveillance areas 104B and 104C,respectively.

In some embodiments, surveillance cameras 106A-D can include a box-stylesecurity camera, a dome security camera, a pan, tilt, and zoom (PTZ)camera, a bullet security camera, a day/night security camera, a thermalsecurity camera, or a wide-dynamic security camera. In some embodiments,as shown in system 100A, security cameras 106A-D may be IP cameras thatcan transmit digital signals using Internet Protocol over network 102.An IP camera may include one or more network interfaces (e.g., a wirednetwork interface and/or a wireless network interface) to connect tonetwork 102. In some embodiments, network 102 can include a local areanetwork (LAN), a wide area network (WAN) such as the Internet, or acombination thereof.

In some embodiments, security cameras 106A-D can capture respectivefields of view within surveillance areas 104A-C and generate videofootage comprising video data representing the captured fields of view.In some embodiments, a video footage can be encoded to include aplurality of video frames and associate each video frame with atimestamp. As will be further described below, security cameras 106A-Dcan be configured to transmit the generated video footage to videomanaging system 120 through network 102. For example, the video footagemay be routed to network 102 directly or through a router device or awireless access point. In some embodiments, security cameras 106A-D maybe connected to a Power over Ethernet (PoE) switch that is connected tonetwork 102. In some embodiments, security cameras 106A-D can beconfigured to route video footage to video analytics system 112, whoseoutputs can be routed to video managing system 120.

In some embodiments, a security camera such as security camera 106A caninclude various hardware and/or software to capture a field of view andgenerate video data including analog and digital signals. For example,the security camera may include one or more image sensors such as ahigh-definition image sensor, a no/low light image sensor, or aspecialized image sensor sensitive to spectrum ranges of light. In someembodiments, the security camera can include a video codec such asH.264, MPEG-4, MPEG-2, HEVC (H.265), etc. to compress and encode videofootage for storage or for transmission to video managing system 120over network 102. In some embodiments, the security camera may bepowered through DC power such as through a Power over Ethernet (PoE)connection or through AC power via an electrical outlet. In someembodiments, the security camera can include one or more networkinterfaces to couple to network 102 such as, for example, an Ethernetinterface, a wireless chip, a mobile communications interface, etc.

In some embodiments, video managing system 120 can be configured toreceive and process one or more video footages generated by securitycameras 106A-D via network 102, as will be further described below withrespect to FIG. 2. In some embodiments, video managing system 120 caninclude one or more servers. For example, these servers may be providedby a cloud architecture to enable reliability of processing andscalability to process video footage from additional security cameras.In some embodiments, video footage can include recorded video or be alive video streamed from security cameras 106A-D to video managingsystem 120. Therefore, video managing system 120 may include a buffer totemporarily store a plurality of video footages while each video footageis being processed.

In some embodiments, video managing system 120 includes a plurality ofwatchlists 122 that control whether and how received video footage is tobe processed. In some embodiments, a watchlist may include one or moreevents of interests represented by one or more corresponding indicatorsand be associated with one or more processing policies, as will befurther described below with respect to FIG. 2. In some embodiments, aprocessing policy can include a collection policy associated withcollecting video of one or more approved events of interest in awatchlist such as watchlist 122A. In some embodiments, the collectionpolicy includes a plurality of surveillance parameters such as one ormore approved locations and one or more approved time periods forcollecting video of the one or more approved events in watchlist 122A.

In some embodiments, video managing system 120 can receive video footage126 from one of surveillance cameras 106A-D and analyze video footage126 to generate a plurality of video segments or clusters of videosegments based on watchlists 122. In some embodiments, video managingsystem 120 can process each video segment or cluster based on one ormore processing policies associated with watchlists 122. In someembodiments, a watchlist may include a plurality of events of interestthat must be matched before the video segment or the video cluster canbe assigned the one or more processing policies associated with thewatchlist. In some embodiments, a watchlist can include a combination ofwatchlists from watchlists 122. In some embodiments, one or more ofwatchlists 122 can associated with one or more specific security camerasor one or more specific surveillance areas. For example, a firstwatchlist from watchlist 122 may be assigned to surveillance area 104Bsuch that the first watchlist should be applied to any video footagederiving from surveillance area 104B, e.g., captured by security camera106C.

In some embodiments, video managing system 120 can include a userinterface to permit a user to access watchlists 122 through a userclient 108. In some embodiments, similar to monitoring client 110, userclient 108 can be provided on a user device (e.g., a tablet, asmartphone, a computer, a desktop, a smartwatch, etc.) that can connectto network 102 to access video managing system 120. In some embodiments,user client 108 may include a standalone application or a thin clientdownloaded on the user device. In other embodiments, the user may accessuser client 108 by accessing a website portal associated with videomanaging system 120. In various embodiments, the user may be promptedwith login credentials before being permitted to configure watchlists122. In some embodiments, video managing system 120 can further controlhow the user may configure watchlists 122 based on an access privilegepossessed by an authenticated user. In some embodiments, a surveillanceuser may be permitted to change retention policies that control how longcollected video is to be stored. However, the surveillance user may bedisallowed from controlling a collection policy as to whether certainvideo content can be collected. As described above, the authorizationuser of authorization system 150 may be permitted the capability toupdate watchlist 152 including the collection policy.

In some embodiments, video managing system 120 can be configured tointerface and communicate with video analytics system 112 to processreceived video footage. In some embodiments, video managing system 120can transmit received video footage to video analytics system 112configured to recognize and detect events of interest within one or moreportions of the video footage. In some embodiments, video analyticssystem 112 can be configured to generate a plurality of video segmentsfrom the video footage based on which events of interest are detectedwithin the video frames of the video footage.

In some embodiments, a video segment includes a portion of the videofootage that includes video frames in which video analytics system 112detected one or more of the same events of interest. In someembodiments, video analytics system 112 can tag the video segment withmetadata including one or more indicators of the one or more events ofinterest detected by video analytics system 112. Accordingly, videoanalytics system 112 can be configured to generate a plurality of videosegments which may include overlapping video content from the videofootage. In some embodiments, video analytics system 112 can beconfigured to discard portions of the video footage in which no eventsof interest are detected. In some embodiments, video analytics system112 can be configured to transmit a plurality of generated videosegments tagged with a plurality of corresponding metadata to videomanaging system 120 via network 102. In some embodiments, videoanalytics system 112 can select one or more frames for each videosegment as representative of prominent features of the video segment.

Returning to video managing system 120, video managing system 120 can beconfigured to generate a plurality of clusters of video segmentsreceived from video analytics system 112 based on the metadata of thevideo segments, according to some embodiments. For example, videomanaging system 120 may aggregate a plurality of video segments sharingat least one indicator in common with a video cluster associated withthe at least one indicator. In another example, video managing system120 may aggregate a plurality of video segments sharing a plurality ofspecific indicators in common with a video cluster associated with thatplurality of specific indicators. In some embodiments, video managingsystem 120 can then match one or more common indicators associated witheach video cluster to indicators in watchlists 122 to identify one ormore watchlists to apply to the video cluster. For example, in responseto determining that the video cluster has two common indicators presentin a first watchlist, video managing system 120 may process the videocluster according to the processing policies specified in the firstwatchlist.

In some embodiments, video managing system 120 can generate a videocollection including one or more video segments or one or more videoclusters based on analyzing video footage 126, as described above.Additionally, video managing system 120 can generate a metadata filethat describe the video collection and include an index of events orfeatures to portions of video footage 126 in which each event or featureis detected. As will be further described below with respect to FIG. 2,video managing system 120 can encrypt video footage 126 stored asencrypted video footage 132 to prevent surveillance users from accessingnon-approved events in video footage 126 without express authorizationfrom authorization system 150.

In some embodiments, video managing system 120 can generate a videocollection 134 for a surveillance user and that includes video segmentsor clusters of approved events as indicated in watchlist 122A associatedwith the surveillance user. For example, video managing system 120 cancopy portions of video footage 126, while unencrypted, to videocollection 134, wherein each portion is determined to include one ormore approved events as indicated in watchlist 122A. In someembodiments, video managing system 120 can encrypt video collection 134using a key possessed by the surveillance user to prevent othernon-approved users from being able to access video collection 134.

In some embodiments, based on a referral policy specified in thewatchlist matching a particular video cluster, video managing system 120may generate a notification or a delayed notification to surveillanceusers through monitoring client 110 to review the video cluster orgenerate no alerts to the user. In some embodiments, based on aretention policy specified in the watchlist matching the particularvideo cluster, video managing system 120 can be configured to discardthe video cluster and not forward the video cluster for long termretention in storage device 130. In other embodiments, video managingsystem 120 can transmit the video cluster to storage device 130 to beretained for a period of time indicated in the retention policy. In someembodiments, however, at least the entirety of video footage 126 isstored as encrypted video footage 132. For example, for law enforcementapplications, portions of video footage 126 selectively captured maypresent a defendant in an unfavorable light without presenting jurieswith the entire video footage 126 to show the full picture. In one ormore examples, under discovery, the defendants counsel could obtain theunencrypted video footage 126 to support counterarguments.

In some embodiments, storage device 130 can be configured to providelong term storage of video data transmitted from video managing system120 to storage device 130. For example, storage device 130 may storevideo footage, video collection, metadata files, and encrypted versionsof each type of video data. As described above, the video segments orvideo clusters may be derived from video footage captured by securitycameras 106A-D. In some embodiments, storage device 130 can include harddrives, solid state disk (SSD) drives, or be a network attached storage(NAS) system. In some embodiments, storage device 130 can be provided byone or more cloud storage devices (e.g., NAS devices) to permit massstorage of large and increasing volumes of video data.

In traditional video data storage solutions, a data storage may eitherbe configured to store video data indefinitely until a user chooses todelete the video data or the video data may be retained for a defaultperiod of time (e.g., one or two months). In the first scenario,indefinitely storing ever increasing volumes of video data is notfeasible and would continually require more storage hardware to storevideo data. In the second scenario, storing video data for a defaultperiod of time is inefficient since portions of video data may be ofhigher priority and should be retained for longer periods of timewhereas other portions of video data may not need to be retained at allor can be retained for brief periods of time. In some embodiments, asdescribed above and will be further described below, storage device 130can be configured to store video collections (e.g., including clustersor video segments) or video footage according to a retention policydetermined by video managing system 120.

In some embodiments, the retention policy may include a retention dateafter which video is to be deleted or include a duration of time such asa number of days, weeks, months, years, etc. For example, a first videocluster may include video content showing specific persons of interest(i.e., example events of interest) and may be assigned a first retentionpolicy of, for example, 3 years. In another example, a second videocluster may include video content showing movement of one or moredetected persons (i.e., example events of interest) and may be assigneda second retention policy of, for example, 9 months. In the secondexample, the second video cluster may not need to be retained for aslong as the first video cluster because users are less likely to needlong term surveillance of all persons in contrast to surveillance ofspecific persons of investigative interest.

In some embodiments, system 100A includes a monitoring client 110 thatpermits the surveillance user to review video collection 134 storingportions of video footage including approved events of interest (asindicated in watchlist 122A). In some embodiments, monitoring client 110can include a central monitoring station or a user device (e.g., alaptop, a computer, a workstation, a smartphone, a tablet, etc.) thatcan connect to network 102 to access video managing system 120. In someembodiments, monitoring client 110 may include a standalone applicationor a thin client downloaded on the user device. In other embodiments,the user may access monitoring client 110 by accessing a website portalassociated with video managing system 120. In various embodiments, theuser may be prompted with login credentials before being permitted toreview portions of video footage authorized for review. In someembodiments, as described above, video managing system 120 may assign animmediate alert to a video cluster or portion of video footage 126. Inthese embodiments, video managing system 120 can be configured togenerate and send the immediate alert to the user through monitoringclient 110 to notify the user to perform immediate video review. Inother embodiments, video managing system 120 may assign a delayed alertto a video cluster. In these embodiments, video managing system 120 maystore video clusters assigned to the delayed alert. The user may reviewthese stored video clusters at a subsequent time via monitoring client110. In some embodiments, users may be permitted to review portions ofvideo footage (e.g., video clusters) stored in storage device 130 usingmonitoring client 110.

FIG. 1B illustrates a system 100B including an authorization system 150for limiting video surveillance executed by surveillance system 103 toauthorized uses, according to some embodiments. System 100B shows manyof the same components as those described above with respect to system100A of FIG. 1A. In contrast to system 100A which showed surveillancecameras 106A-D, system 100B shows that video managing system 120 can beconfigured to process a video stream or a video footage captured byother types of surveillance cameras 106E-I, according to someembodiments. Video managing system 120 can be configured to performvideo surveillance across a plurality of surveillance areas 104D-F,according to some embodiments.

In some embodiments, in addition to security cameras 106A-D which are IPcameras described with respect to FIG. 1A, video managing system 120 canbe configured to process video footage captured by an analog camerasystem such as analog cameras 106E-F (also referred to as closed-circuittelevision (CCTV) cameras) monitoring surveillance area 104D. In theseembodiments, analog cameras 106E-F can be physically coupled to adigital video recorder (DVR) 126 through respective cables (e.g.,coaxial cables). In some embodiments, DVR 126 can be configured toconvert the analog video captured by analog cameras 106E-F into digitalvideo footage. In some embodiments, DVR 126 can be coupled to network102 through a network device (e.g., a router, a switch, etc.) andconfigured to transmit the digital video footage to video managingsystem 120 for further processing. In some embodiments, video footagecaptured by analog cameras 106E-F can be temporarily buffered in DVR126, whereas the video footage transmitted to video managing system 120may be stored in storage device 130 for longer periods of time dependingon whether certain criteria are met, as described above with respect toFIG. 1A. For example, the video footage stored in DVR 126 may be storedfor a default period of time (e.g., 8 hours, 1 day, 1 week, etc.) thatpermits DVR 126 enough time to offload the video footage to videomanaging system 120. In some embodiments, DVR 126 can be configured tostore the captured video footage on its internal storage device untilthe video footage is forwarded to video managing system 120, after whichthe video footage may be deleted or marked for deletion.

In some embodiments, in addition to or alternative to security cameras106A-D described with respect to FIG. 1A, video managing system 120 canbe configured to process video footage captured by edge video devicessuch as edge camera 106G. Edge camera 106G can be an IP cameraco-located with video analytics capabilities and optionally data storagefunctionality. For example, edge camera 106G may include an analyzer 140configured to provide near real-time video analytic functionalityincluding some or all of those provided by video analytics system 112,as described above with respect to FIG. 1A. Like video analytics system112, analyzer 140 may be configured to analyze the video footage todetermine whether one or more video segments include one or more eventsof interest (e.g., one or more targets) as well as tag indicators toeach video segment to indicate detected events of interest. In someembodiments, edge camera 106G can store one or more of watchlists 122specific to edge camera 106G or surveillance area 104E and the one ormore stored watchlists may indicate the one or more events of interestto be detected. In some embodiments, the one or more watchlists can bereceived from authorization system 140. In some embodiments, analyzer140 can transmit a plurality of video segments and one or moreassociated indicators to video managing system 120 for furtherprocessing. In some embodiments, analyzer 140 can also perform some orall of the functionality as that provide by video managing system 120and described above. In these embodiments, analyzer 140 may generateclusters of video segments, match video clusters to one or morewatchlists 122 stored locally on edge camera 106, and process a videocluster according to the one or more processing policies in a watchlistmatching the video cluster.

For example, as described above with respect to FIG. 1A, a watchlist mayinclude one or more classes of objects or one or more specific objectsor persons. The watchlist may also include one or more processingpolicies including a collection policy. The processing policies may alsoinclude a retention policy and/or a review policy. In response toanalyzer 140 determining that, for example, a video segment or a clusterof video segments is detected to include a target and that meets thereview policy, analyzer 140 may alert a user through monitoring client110 based on the review policy and/or transmit the video segment orvideo cluster over network 102 for long term storage depending on theretention policy associated with the watchlist.

In some embodiments, edge camera 106G can include a regulator 144configured to control and enforce compliance policies for videosurveillance performed by edge camera 106G. In particular, regulator 144can provide one or more of the same control and enforcement functionsprovided by video managing system 120. For example,

In some embodiments, video footage captured by edge camera 106G can bestored in data store 142. For example, a plurality of video segments ofthe video footage may be temporarily buffered in data store 142 whileanalyzer 140 runs video analytics on each video segment. In someembodiments, data store 142 may be configured to store the videosegments for a default period of time (e.g., 8 hours, 1 day, 3 days, 1week, etc.). In some embodiments, once analyzer 140 finishes processinga video segment, analyzer 140 can be configured to delete the videosegment from data store 142 or to mark the video segment for deletion.

In some embodiments, the types of analytics functionality as well ofavailable processing speed of edge camera 106G may depend on power, heatdissipation, and size/weight constraints of edge camera 106G. Oneadvantage of edge camera 106G is that unlike IP cameras such as securitycameras 106A-D, edge camera 106G does not need to be continuouslyconnected to network 102 to route captured video footage to videomanaging system 120 for further processing. In some embodiments,analyzer 140 on edge camera 106G can perform fine grained filteringbased on one or more watchlists or perform a first filtering step toreduce the amount of video footage that is transferred over to network102 to be further processed by video managing system 120.

In some embodiments, in addition to or alternative to security cameras106A-D described with respect to FIG. 1A, video managing system 120 canbe configured to process video footage captured by IP cameras 106H-Ithat are not directly connected to network 102. In some embodiments, IPcameras 106H-I may be directly connected to a network video recorder(NVR) 128 or indirectly connected to NVR 128 through switch 127. Forexample, IP cameras 106H-I may be connected to NVR 128 through a wiredconnection (e.g., Ethernet cables) or a wireless connection (e.g.,WiFi). In some embodiments, switch 127 may include a power-over-Ethernet(PoE) switch that provides both power and data communication to IPcameras 106H-I.

In some embodiments, NVR 128 can be configured to record video footagecaptured by IP cameras 106H-I and store captured video footage. In someembodiments, like DVR 126, NVR 128 can be configured to forward storedvideo footage to video managing system 120 for further processing. Forexample, NVR 128 may transmit the video footage via a router connectedto switch 127. In some embodiments, NVR 128 may function as temporarystorage for captured video footage, whereas video footage transmitted tovideo managing system 120 may be retained for long term storage onstorage device 130 depending on whether certain criteria are met, asdescribed above with respect to FIG. 1A. For example, NVR 128 may storecaptured video footage for a short period of time (e.g., 1 hour, 8hours, 1 day, etc.) whereas video footage stored on storage device 130can be retained for much longer periods of time (e.g., 6 months, 9months, 1 year, 7 years, etc.). In some embodiments, NVR 128 can beconfigured to store the captured video footage on its internal storagedevice until the video footage is forwarded to video managing system120, after which the video footage may be deleted or marked fordeletion.

In some embodiments, functions provided by video analytics system 112may be implemented by video managing system 120. For example, videomanaging system 120 may include a video analytics system 124 which maycorrespond to video analytics system 112. In some embodiments, videoanalytics system 124 may include one or more objects databases or facedatabases to permit video analytics system 124 to detect classes ofobjects or motion, specific objects or persons, or a combinationthereof.

FIG. 2 illustrates a block diagram 200 showing examples for how a videofootage 202 is processed to generate video collections 248A-B accessibleby respective users 250A-B, according to some embodiments. For ease ofreference, the following descriptions may refer to one or morecomponents of system 100A, as described above with respect to FIG. 1A.As described above with respect to FIG. 1A, each video collection caninclude captured video content including only events of interestsapproved for collection for the user.

As shown in diagram 200, video footage 202 can span a time period 204from time 0 to T. Video managing system 120, with assistance from videoanalytics system 112, can detect a plurality of events (e.g., indicatingtargets P, A, and B) in portions of video footage 202. In someembodiments, the plurality of events are indicated in watchlist 212. Forexample, the line markings 206A-C show time periods from time period 204for respective events A, B, and C during which each respective event isdetected. For example, line marking 206C shows periods of time where oneor more persons P are detected. Similarly, line markings 206A-B showperiods of time where in video footage 202 specific persons A and B,respectively, are detected.

In some embodiments, video managing system 120 can be configured togenerate a plurality of video segments based on the events of interestin watchlist 212. For example, video managing system 120 may generate avideo segment as one or more video frames that are detected to includean event in watchlist 212. For example, video segments 208 may includevideo segments 208A-C for event P and corresponding to time periodsshown by line markings 206C. Similarly, video segments 208 may includevideo segments 208D-E and video segments 208F-G for events A and B,respectively, and corresponding to the time periods shown in linemarkings 206A-B, respectively. In some embodiments, video managingsystem 120 can copy one or more portions of video footage 204 togenerate the video segments.

In some embodiments, video managing system 120 can generate videosegments 210 that can each be tagged with one or more indicators ofevents of interest. For example, video segments 210A, 210F, and 210H maybe generated and associated with only event P; video segments 210B and210E are associated with both events B and P; video segments 210D and210G are associated with both events A and P; and video segment 210C isassociated with events P, A, and B. In some embodiments, video managingsystem 120 can generate video segments 210 based on video footage 202 orfrom video segments 208. In some embodiments, video managing system 120can generate clusters of video segments in which each cluster shares oneor more indicators in common. For example, video managing system 120 maygenerate a video cluster for target A and include, for example, videosegments 208D and 208E.

In some embodiments, video managing system 120 can select one or moreframes from each video segment to represent the salient features of eachvideo segment. Then, video managing system 120 can analyze the one ormore selected frames to detect one or more features to associate witheach video segment.

In some embodiments, to permit faster and more flexible retrieval andanalysis of certain events of interest, video managing system 120 cangenerate metadata file 220 based on video footage 202 and video segments208 and/or 210. In some embodiments, metadata file 220 includes an indexof a plurality of events (e.g., as indicated in watchlist 212) and thatassociates each event with one or more portions of video footage 202 inwhich that event is detected. In some embodiments, the index canassociate an event with one or more video frames, one or more videosegments from video segments 208-210, or one or more time periods fromvideo footage 202. In some embodiments, metadata file 220 can alsoinclude an index of detected features and associated video segments inwhich each features is detected. By keeping these indices, metadata file220 may permit authorized users to search and retrieve video segmentsassociated with desired targets and/or events having certain features.

In some embodiments, user 250A (i.e., user A) may be associated withwatchlist 240A that specifies events A and B and respective collectionpolicies. In some embodiments, a collection policy for an event caninclude one or more surveillance parameters that specify criteria forcollecting video content of that event. In some embodiments, theparameters can include one or more of a location, a time period, andwhether that event's video can be viewed by user 250A. For example, thecollection policy for event B may indicate approved video collection ofB during time period 204 of video footage 202 and the collection policyfor event A may indicate disapproved video collection of A during timeperiod 204 of video footage 202. Accordingly, video managing system 120may copy one or more portions of video footage 202 to video collection242A that includes approved event B and does not include disapprovedevent A. For example, video collection 242A may include copies of videosegments 210B and 210E from video segments 210. In some embodiments,video managing system 120 can determine the portions to include in videocollection 242A based on searching metadata file 220 and identifying oneor more of video segments 208-210.

In some embodiments, similar to generation of metadata file 220 forvideo footage 202, video managing system 120 can generate metadata file244A associated with generated video collection 242A. In someembodiments, metadata file 244A can include an index of approved events(e.g., event B) and link to one or more video segments, clusters, orvideo frames in which an approved event is detected. For example, themetadata file 244A may list video segments 210B and 210E in videocollection 242A and associated these segments with event B. In someembodiments, metadata file 244A can be generated based on metadata file220. For example, metadata file 244A may omit or discard portions ofmetadata file 220 that include reference to disapproved event A.

As described above with respect to FIG. 1A, watchlists associated withdifferent users may permit each user to access only video content ofapproved targets with respect to each user. In some embodiments, user250B (i.e., user B) may be associated with watchlist 240B that specifiesevents A and B and respective collection policies. For example, thecollection policy for events A and B may indicate approved videocollection of A and B during time period 204 of video footage 202.Accordingly, video managing system 120 may copy one or more portions ofvideo footage 202 to video collection 242B that includes approved eventA or B. For example, video collection 242B may include copies of videosegments 210B-210D, 210E, and 210G. In contrast to video collection242A, video collection 242B can include portions of video footage thatdepict target A. Similar to generation of metadata file 244A for videocollection 242A, video managing system 120 can generate metadata file244B associated with generated video collection 242B. For example,metadata file 244B can include an index of approved events (e.g., eventA and B) and link to one or more video segments, clusters, or videoframes in which an approved event is detected.

As described above with respect to FIG. 1A, once video footage 202 hasbeen processed (e.g., to generate video collections 242A and 242B),video managing system 120 can generate encrypted video footage datausing an encryption key 230. In some embodiments, video managing system120 can encrypt video footage 202, video collection of segments 208-210,and metadata file 220 to generate encrypted video footage 224, encryptedvideo collection 226, and encrypted metadata file 228. In someembodiments, encryption key 230 may be managed and secured byauthorization system 260 to prevent surveillance users that access videomanaging system 120 from being able to decrypt any of encrypted videofootage data 222. In some embodiments, once encrypted video footage data222 is generated, video managing system 120 may irrecoverably deleteencryption key 230. In some embodiments in which video managing system120 implements one or more asymmetric encryption algorithms, encryptionkey 230 may be a public key of a key pair managed by authorizationsystem 260. Once encrypted, video managing system 120 cannot decryptencrypted video footage data 222 without access to a private key of thekey pair managed by authorization system 260.

In some embodiments, video collections 242A-B may be stored unencryptedon storage device 130 to permit surveillance users 250A-B to access andview video content captured in video collections 242A-B. In someembodiments, to increase security to limit access to a video collectiononly to those users permitted to access the video collection, each videocollection and associated data may be encrypted by user-specific keys.For example, video managing system 120 may encrypt video collection 242Aand metadata file 244A with a key 232A possessed by user 250A togenerate encrypted video collection data 246A including encrypted videocollection 248A and encrypted metadata file 249A. Similarly, videomanaging system 120 may encrypt video collection 242B and metadata file244B with a key 232B possessed by user 250B to generate encrypted videocollection data 246B including encrypted video collection 248B andencrypted metadata file 249B.

FIG. 3 illustrates a block diagram 300 showing an example for how anupdated watchlist 308 (e.g., watchlist B) of a user can be applied to apreviously captured video footage, according to some embodiments. Forease of reference, the following descriptions may refer to one or morecomponents described above with respect to FIG. 1A. As shown in diagram300, the previously captured video footage may be encrypted and storedin encrypted video footage data 310. For example, as described withrespect to FIG. 2, video managing system 120 may generate encryptedvideo footage data 310 including encrypted video footage 312, encryptedvideo collection 314, and encrypted metadata file 316.

In some embodiments, to apply updated watchlist 308 to the previouslycaptured video footage, video managing system 120 may first need toaccess decrypted video data. Accordingly, video managing system 120 canapply a key 302A (key M1) received from authorization system 150 todecrypt encrypted video footage data to generate decrypted video footagedata 318. Decrypted video footage data 318 includes decrypted videofootage 320, decrypted video collection 322, and decrypted metadata file324, which are decrypted versions of encrypted video footage 312,encrypted video collection 314, and encrypted metadata file 316.

In some embodiments, a previous watchlist 306 (i.e., watchlist A)associated with the user may have been applied on the previouslycaptured video footage to generate encrypted video collection data A 340including encrypted video collection A 342 and encrypted metadata file A344, as described above with respect to FIG. 2. For example, encryptedvideo collection A may include encrypted copies of portions of the videofootage that include one or more approved targets specified in watchlistA 306. In some embodiments, video managing system 120 can apply a key A304A associated with the user to decrypt encrypted video collection dataA to generate decrypted video collection data A 346 including decryptedvideo collection A 348 and decrypted metadata file A 350.

In some embodiments, video managing system 120 can apply updatedwatchlist B 308 to decrypted video collection data A 346 and decryptedvideo footage data 318 to generate video collection data B 354 includingvideo collection B 356 and metadata file B 358. In some embodiments,video collection B 356 can include one or more copies of portions of thevideo footage (e.g., one or more video segments, one or more videoframes, etc.) that include one or more events approved in updatedwatchlist 308 and that do not include any disapproved event in updatedwatchlist 308. In some embodiments, video managing system 120 can applyupdated watchlist B 308 directly to decrypted video footage data 318.For example, video managing system 120 may search an events or featuresindex in decrypted metadata file 324 to identify one or more portions ofdecrypted video footage 320 or one or more video segments in decryptedvideo collection 322 that meets the criteria of updated watchlist B 308.In some embodiments, video managing system 120 can modify decryptedvideo collection A 348 to generate video collection B 356.

For example, as shown and described with respect to FIG. 2, watchlist A306 and updated watchlist B 308 may correspond to watchlists 240A and240B, respectively. Accordingly, decrypted video collection A 348 mayinclude video segments 210B and 210E including event B and generatedvideo collection B 356 may additional video segments as shown in videocollection 242B to include video segments showing event A. In a lawenforcement scenario, event B may represent an arrested individual andevent A may represent the individual's attorney. If both events A and Bare captured in video, law enforcement may not be permitted to view orlisten in on such video due to attorney-client privileges. In the rarecircumstance that the courts determines that the attorney-clientprivilege does not apply, a surveillance user from law enforcement maybe granted an updated watchlist 308 to view additional video content inwhich both events A and B are detected in the same time frame.

Similarly, video managing system 120 can generate metadata file B 358that includes an index of events indicated updated watchlist B 308. Insome embodiments, video managing system 120 can generated metadata fileB 358 based on decrypted metadata file 324. In some embodiments,metadata file B 358 includes change records 359 that indicate a changebetween decrypted video collection data A 346 and video collection dataB 354 to provide an audit record.

In some embodiments, video managing system 120 can further encryptgenerated video collection data B 354 using key B 304B associated withthe user to generate encrypted video collection data B includingencrypted video collection B 362 and encrypted metadata file B 364corresponding to encrypted versions of video collection B 356 andmetadata file B 358, respectively. In some embodiments, key B 304B maybe the same as key A 304A.

In some embodiments, should updated watchlist B 308 include events notoriginally detected in the video footage, video managing system 120 canbe configured to generate updated video footage data 326 includingupdated video collection 327 and updated metadata file 328. In someembodiments, to re-secure decrypted video footage data, video managingsystem 120 can apply key M2 302B to generate encrypted video footagedata 330. In some embodiments, encrypted video footage data 330 caninclude encrypted video footage 336, encrypted updated video collection332, and encrypted updated metadata file 334, which correspond toencrypted versions of decrypted video footage 320, updated videocollection 327, and updated metadata file 328. In some embodiments, keyM2 302B can be provided by authorization system 150 and is differentthan the original key used to generated encrypted video footage data310. In some embodiments, once encrypted, video managing system 120cannot decrypt encrypted video footage data 330 using key M2 302B whenimplementing an asymmetric encryption scheme.

FIG. 4 illustrates a block diagram of a system 400 for authorizing andexecuting video surveillance, according to some embodiments. System 400includes an imaging system 402, a video analytics system 406, a videomanaging system 420, an authorization system 470, and a storage device440. In some embodiments, video managing system 420, video analyticssystem 406, authorization system 470, and storage device 440 maycorrespond to similarly named video managing system 120, video analyticssystem 112, authorization system 150, and storage device 130,respectively. In some embodiments, as described above with respect toFIG. 1B, video managing system 420 and video analytics system 406 can beimplemented in an integrated video managing system.

In some embodiments, imaging system 402 can include one or more securitycameras such as security cameras 106A-D or security cameras 106E-I asdescribed above in FIGS. 1A and 1B, respectively. For example, imagingsystem 302 may include an IP camera configured to communicate with videomanaging system 420 through an IP network. In some embodiments, imagingsystem 402 can include a surveillance network of security cameras. Insome embodiments, a security camera in imaging system 402 can generatevideo footage 404 that includes video content captured within a field ofview of an area under surveillance by the security camera. In someembodiments, video footage 404 can be stored in a video format torepresent a plurality of video frames tagged by metadata such astimestamps indicating when the video content was captured. Accordingly,video footage 404 can include a plurality of video frames. In someembodiments, imaging system 402 can tag video footage 404 with metadataindicating an identifier of a security camera generating video footage404 or one or more indicators of a location of the area undersurveillance by the security camera.

In some embodiments, video footage 404 can be stored and transmittedaccording to one or more video formats such as H.264, MJPEG, MPEG-4,MPEG-2, HEVC (H.265), etc. In some embodiments, imaging system 402 canbe configured to transmit video footage 404 to video managing system 420or video analytics system 406 through a network (e.g., network 102 ofFIG. 1A). In some embodiments, video footage 404 can be transmitted as alive video stream in real time or near real time. In other embodiments,video footage 404 can be transmitted as recorded video, as describedabove with respect to security cameras 106A-I in FIGS. 1A-B.

In some embodiments, video managing system 420 receiving video footage404 can be configured to request video analytics system 406 to parsevideo footage 404 into a plurality of video segments 408. For example,video managing system 420 may forward the received video footage 404 tovideo analytics system 406 through a network (e.g., network 102). Inother embodiments, video footage 404 may be directly transmitted tovideo analytics system 406 for processing.

In some embodiments, video analytics system 406 can be configured togenerate a plurality of video segments 408 based on the received videofootage 404 such that each video segment includes related video content.In some embodiments, video analytics system 406 can analyze each videosegment to select one or more video frames that are representative ofeach video segment. In some embodiments, video analytics system 406 canfurther analyze the one or more video frames to extract one or morefeatures to associate with the video segment. For example, the one ormore features may include colors, shapes, objects, classes of objects,etc.

In some embodiments, video analytics system 406 can be configured toapply video analytics to detect one or more events of interest withinvideo footage 404 and generate a video segment of video frames that aredetected to include the one or more events of interest. In someembodiments, each video segment include a plurality of video frameswithin a predefined period of time that include one or more events ofinterest in common as detected by video analytics system 406. In someembodiments, video analytics system 406 can be configured to detectevents of interests in the video frames of video footage 404 based on aplurality of watchlists supplied by video managing system 420 orauthorization system 470.

In some embodiments, as shown in video managing system 420, a watchlist424 can include one or more events of interest such as event of interest426 that is associated with one or more processing policies 428A-B. Aswill be further described below, processing policies 428A-B includeinformation that controls how video content that contains event ofinterest 426 should be processed. For example, processing policies428A-B may control whether the video content should be referred to usersfor immediate review, whether the video content can be retained onstorage device 240 for long term storage, and if so, for how long thevideo content should be retained on storage device 240. Additionally,processing policies 428A-B may include a collection policy that dictateswhich portions of video footage 404 are viewable by a surveillance user.In some embodiments, the collection policy associated with event 426 caninclude one or more surveillance parameters such as one or more approvedlocations for surveillance and one or more time periods forsurveillance.

In some embodiments, watchlist 424 can include one or more classes ofobjects or object motion. For example, event of interest 426 may includea presence or motion within a field of view of the following: one ormore persons, one or more vehicles, one or more aircraft, one or morebusses, one or more trucks, one or more bicycles, one or more weapons,luggage, one or more backpacks, one or more hats, etc. In otherexamples, event of interest 426 may be a class of motion or activitiessuch as a running person, bending down, a falling person, etc.

In some embodiments, watchlist 424 can include one or more specificobjects or activities within a field of view. For example, event ofinterest 426 may include a specific vehicle identified by a licenseplate, a specific type of weapon, a person performing a specificactivity or motion, etc. In some embodiments, event of interest 426 caninclude one or more specific persons of interest. In these embodiments,reference images of a person of interest may need to be supplied by theuser.

In some embodiments, watchlist 424 can include one or more objects orpersons characterized by one or more descriptors. For example, event ofinterest 426 may indicate one or more vehicles of a specific make,model, color, or a combination thereof, each of which are exampledescriptors. In another example, event of interest 426 may indicate oneor more persons meeting criteria required by one or more descriptors. Inthese example, the one or more descriptors for persons may include adesignated hat color, a hair color, upper garment type or color, lowergarment type or color, a height range, a gender, accessories likeglasses or rings, a tattoo, etc. In some embodiments, watchlist 424effectively can act as a filter on captured video content such that onlyvideo content that includes the characteristics of unknown persons ofinterest are retained and subsequently forwarded to authorized users formanual review.

In some embodiments, watchlist 424 can include a whitelist or ablacklist. In some embodiments, the whitelist includes a plurality ofevents of interest whose captured video content should be retained orreferred for review. For example, in a public or national securitycontext, the whitelist may include a plurality of individuals ofinvestigative interest. For example, common whitelists may include theTerrorist Screening Centers Terrorist Watchlist, the State Department'sArms Export Control Act (AECA) Debarred List and U.S. Denied PersonsList, the FBI's Ten Most Wanted List, or Wanted Persons listedmaintained by the FBI for whom open judicial warrants exist. In othercontexts such as at fundraising events in politics, sports, or atuniversities, a whitelist may include prominent donors or family membersof these donors which fundraisers may be particularly interested intracking. In yet other contexts such as in a commercial context, thewhitelist may include a list of VIP customers or important personnelthat should be tracked.

In contrast, a blacklist may include a plurality of events of interestwhose captured video content should not be retained, according to someembodiments. For example, in a commercial context, the blacklist mayinclude employees expected to be regularly present at surveilledlocations and whose image as captured by security cameras should not beretained. In a residential context such as at an apartment complex,security personnel may be concerned with protecting privacy of residentsand may generate the blacklist of residents at the apartment complexsuch that video content captured of residents do not require review,collection, nor storage. In this context, security guards are moreconcerned with monitoring unknown persons, which may be indicated in awhitelist.

Accordingly, video analytics system 406 can run video analytics on oneor more video frames of video footage 404 to generate a plurality ofvideo segments 408 based on events of interest identified in one or morewatchlists including, for example, watchlist 424. As described above, anevent of interest may include a presence or motion of a class of objectssuch as persons, vehicles, weapons, etc. The event of interest may alsoinclude a presence or motion of a specific type of object or personscharacterized by one or more descriptors or specific objects or personsof interest. For example, such an event of interest may include a redvehicle, a vehicle with a specific license plate, a particular make ormodel of vehicle, etc.

In some embodiments, video analytics system 406 can be configured toimplement one or more object detection algorithms to detect one or moreevents of interest including a presence of a class of objects orspecific objects within video frames of video footage 404. In someembodiments, an object detection algorithm can include one or moremachine learning algorithms such as Convolution Neural Networks (CNNs),Region-based CNN (R-CNN), Fast R-CNN, Faster R-CNN, YOLO, etc. In someembodiments, reference images for objects or extracted features forobjects can be stored in objects database 410. In some embodiments,video analytics system 406 can run one or more object detectionalgorithms to generate feature results and compare the feature resultswith those stored in objects database 410 to detect a class of objectsor a specific object.

In some embodiments, video analytics system 406 can be configured toimplement one or more facial recognition algorithms to detect a presenceof one or more persons or to detect one or more specific persons ofinterest within video frames of video footage 404. In some embodiments,the one or more facial recognition algorithms can be configured todetect faces, extract features from a detected face, and match theextracted features to a face database 412 of face features to identifyone or more specific persons. In some embodiments, the one or morefacial recognition algorithms can include one or more machine learningalgorithms such as those described above for object detection and/orprincipal component analysis (PCA) using eigenfaces, linear discriminantanalysis (LDA), elastic bunch graph matching using the Fisherfacealgorithm, a hidden Markov model, multilinear subspace learning usingtensor representation, neuronal motivated dynamic link matching, or acombination of two or more algorithms. In some embodiments, videoanalytics system 406 can be configured to train one or more of itsmachine learning algorithms based on a plurality of reference images ofa person of interest to permit detection of that person in one or morevideo frames of video footage 404.

In some embodiments, video analytics system 406 can be configured toapply one or more object recognition or facial recognition algorithms onvideo footage 404 to segment video footage 404 into a plurality of videosegments 408. In some embodiments, each video segment can include aplurality of video frames that have been determined by video analyticssystem 406 to include an event of interest. In these embodiments, videoanalytics system 406 can be configured to tag the video segment withmetadata indicating presence of the detected event of interest. In someembodiments, the metadata may include one or more indicators for one ormore detected events of interest and one or more correspondingconfidence scores.

In some embodiments, during review of video footage 404, video analyticssystem 406 can initiate a start of a video segment associated with anevent of interest in response to detecting the event of interest in thevideo footage 404. In some embodiments, while the event of interest isdetected in following video frames of video footage 404 within apredetermined period of time, additional video frames may be added tothe video segment. In some embodiments, video analytics system 406 candetermine an end of the video segment in response to determining thatthe event of interest has not been detected in video footage 404 withina period of time. In some embodiments, to add buffer time to the videosegment, video analytics system 406 can be configured to add one or moretime buffers so that the delimited video segment begins a first periodof time (e.g., minutes) before the event of interest is detected andends a second period of time (e.g., minutes) after the event of interestis no longer being detected.

In some embodiments, object and facial recognition algorithms operate ona probabilistic basis and provide a confidence score (e.g., 80%) foreach indicator of a detected event of interest indicating how likelythat event of interest was accurately classified or detected. Asdescribed above, an event of interest may indicate a presence of a classof objects or specific objects or persons of interest. Therefore, avideo segment that is tagged with three indicators for three detectedevents of interest may be tagged with three corresponding confidencescores. In this example, the three indicators may include a firstindicator of presence of one or more persons detected in the videosegment, a second indicator of a presence of a specific person ofinterest detected in the video segment, and a third indicator of apresence of one or more weapons detected in the video segment. In someembodiments, video analytics system 406 can be configured to tag each ofvideo segments 408 with one or more indicators having a confidence scoreabove a predefined threshold value. In other words, video analyticssystem 406 may be configured to filter video segments from beingtransmitted to video management system 420 depending on whether theconfidence score is above a predefined threshold value.

In some embodiments, authorization system 470 can be configured tocontrol and enforce compliance of video surveillance performed by videomanaging system 420 in accordance with video collection policies. Insome embodiments, authorization system 470 can include a controlprocessor 479 that configures and manages each of events database 472,keys database 478, and video collection database 484.

In some embodiments, authorization system 470 can include an eventsdatabase 470 that associates each event 474 with one or more policies476. In some embodiments, events database 470 can store one or morewatchlists approved by an authorization user having access toauthorization system 470. For example, a surveillance user may submitrequested additions or deletions from watchlist 424 and theauthorization user may approve the request to configure events database472. In some embodiments, control processor 479 can forward an approvedwatchlist from events database 472 for use by video managing system 420.Accordingly, in some embodiments, only authorized users of authorizationsystem 470 can control a collection policy for an event of interest tocontrol surveillance parameters for collecting video content of thatevent of interest.

In some embodiments, authorization system 470 can include a keysdatabase 478 that includes one or more keys 482 for each video ID 480indicating a video footage or a security camera generating the videofootage. In some embodiments, a key 482 for video ID 480 can be used byvideo managing system 420 to encrypt video footage 404 identified byvideo ID 480 such that surveillance users of video managing system 420cannot access video footage 404 once encrypted. Control processor 479can control when video managing system 420 is permitted to decryptencrypted video footage data 441. As described above with respect toFIG. 3, to permit an surveillance user to decrypt, for example,encrypted video footage 450, control processor 479 can transmit adecryption key (e.g., a private key in a key pair) associated with videoID 480 to video managing system 420. In some embodiments, controlprocessor 479 can also transmit a new public key of a new key pairassociated with video ID 480 to video managing system 420 to re-encryptdecrypted video footage data. Accordingly, video managing system 420would need to request access to a private key stored with video ID 480to access encrypted video footage data 441.

In some embodiments, authorization system 470 can include a videocollection database 484 that includes one or more metadata files 488 foreach video ID 486 indicating a video footage or a security cameragenerating the video footage. In some embodiments, metadata file 484 mayby generated by video managing system 420 for video footage 404, asdescribed above with respect to FIGS. 2-3. For example, metadata file484 may include an index of events of interest and portions of the videofootage at which each event is detected. Metadata file 484 may alsoinclude information that describe the video segments or video clustersof a video collection generated from video footage 404. In someembodiments, authorization system 470 can be configured to transmitmetadata file 488 to video managing system 420 to permit a surveillanceuser authorized by authorization system 470 to analyze previouslyencrypted video footage data 441. For example, the indexes stored inmetadata file 488 may permit video managing system 420 to moreefficiently search for events of interest in a decrypted video footage.

In some embodiments, video managing system 420 can be configured toperform targeted video surveillance processing and collection of videofootage 404 based on one or more watchlists such as watchlist 424 and ascontrolled by authorization system 470. In some embodiments,communications between video managing system 420 and authorizationsystem 470 can be encrypted to prevent man in the middle attacks onsensitive data such as keys 482. In some embodiments, to process videofootage 404, video managing system 420 can include a user interface 422,a clustering component 430, a video analytics (VA) interface 432, aretention component 434, and a command parser 436. In some embodiments,video managing system 420 can be implemented on one or more serverscentrally located or distributed across a network. In other embodiments,video managing system 420 can be implemented on one or more cloudservers in a cloud platform.

In some embodiments, VA interface 432 is configured to communicate withvideo analytics system 406. For example, video managing system 420 mayreceive video footage 404 from imaging system 404 and VA interface 432may route video footage 404 to one or more video analytics systems suchas video analytics system 406. In some embodiments, VA interface 432 canthen receive video segments 408 tagged with metadata as generated byvideo analytics system 408. In some embodiments, VA interface 432 can beconfigured to transfer a copy of a plurality of watchlists to videoanalytics system 406 to control which types of events of interest videoanalytics system 406 is to detect. As described above, watchlists suchas watchlist 424 include a plurality of events of interest such as eventof interest 426. In some embodiments in which an integrated videomanaging system 405 includes video analytics system 406, VA interface432 may be an optional component.

In some embodiments, user interface 422 can be configured to permitsurveillance users to access watchlist 424 stored on video managingsystem 420 as well as to access, from storage device 440, encryptedvideo collection data 460 associated with a surveillance user. In someembodiments, user interface 422 can be provided as a web application ora web portal to the user who can access user interface 422 through auser device capable of connecting to a network (e.g., the Internet)accessed by video managing system 420. In some embodiments, userinterface 422 can prompt a user to enter login credentials toauthenticate the user before permitting access to watchlist 424 or toview video content of encrypted video collection data 460 stored onstorage device 440. In some embodiments, user interface 422 can beconfigured to restrict how the user can interact with watchlist 424based on an access privileged possessed by the user. For example, userinterface 422 may permit a first user having a read-only access to viewevent of interest 226 and associated processing policies 428A-B, butdoes not permit any changes to watchlist 424. In another example, userinterface 422 may permit a second user having a limited access to assigna restricted set of processing policies to events of interest. Inanother example, user interface 422 may permit a third user havingaccess to request changes to collection policy to authorization system470. For example, the third user may be granted a court order, awarrant, or target consent to add or remove events of interest fromwatchlist 424. In some embodiments, video managing system 420 can beconfigured to forward the request to authorization system 470. Uponapproval by authorization system 470, video managing system 420 canupdate watchlist 424.

In some embodiments, clustering component 430 can be configured togenerate a plurality of clusters of related video segments from videosegments 408 received from video analytics system 406. In someembodiments, clustering component 430 can be configured to aggregate aplurality of video segments that have one or more indicators in commoninto a video cluster. For example, clustering component 430 may comparemetadata between video segments 408 to determine which video segmentshave one or more indicators in common. In some embodiments, each videocluster can be associated with a unique combination of a plurality ofindicators. Therefore, it is possible for two video clusters to includethe same video segment if that video segment tagged with one or moreindicators contained in both of the two video clusters.

In some embodiments, command parser 436 can be configured to generate avideo collection of video segments or clusters based on received videofootage 404 by detecting a plurality of events indicated byauthorization system 470. In some embodiments, as described above withrespect to FIGS. 2-3, command parser 436 can encrypt video footage data441 for storing in storage device 440 using a first key (e.g., key 482)associated with video footage 404. For example, encrypted video footagedata 441 stored on storage device 440 can include encrypted videofootage 450, encrypted metadata file 452, and encrypted video collection441. As shown, encrypted video collection 441 may include a plurality ofvideo segments 444 or clusters of segments and one or more associatedindicator 446 for the detected event(s) of interest. In someembodiments, command parser 436 can also apply one or more processingpolicies 428A-B to store video footage data based on watchlist 424. Forexample, storage device 440 may store retention data 448 with encryptedvideo footage data 441 that controls a storage retention period.

In some embodiments, as described above with respect to FIG. 1A, one ormore processing policies 428A-B may include a referral policy, aretention policy, a collection policy, or a combination thereof. Forexample, the referral policy may indicate that the video content bereferred to human analyst immediately (i.e., an immediate alert),flagged for later review (i.e., a delayed alert), or not alerted at all.For example, the retention policy may indicate whether the video contentis authorized for retention and should be retained or deleted. In anexample, the retention policy may indicate a period of time (e.g., anumber of days, weeks, months, years, etc.) to retain video content instorage device 440. In an example, the retention policy may indicate aretention date to retain the video content in storage device 440, afterwhich the video content should be deleted. For example, the collectionpolicy may indicate one or more surveillance parameters for an approvedevent of interest.

In some embodiments, command parser 436 can be configured to generate avideo collection for a surveillance user. The video collection caninclude a copy of a plurality of portions of video footage 404 that aredetected to include one or more approved events of watchlist 424associated with the surveillance user. In some embodiments, commandparser 436 can generate a metadata file associated with the generatedvideo collection and that describe the video segments or clusters storedin the video collection. In some embodiments, the metadata file includesan index of events and that associates each event with one or moreportions of video footage 404 at which that event can be viewed and isdetected. In some embodiments, command parser 436 can encrypt and storethe video collection 462 and associated metadata file 464 as encryptedvideo collection data A 460 on storage device 440. In some embodiments,command parser 436 can apply an encryption key possessed by thesurveillance user to generate encrypted video collection data 460 suchthat other users that do not possess the encryption key cannot accessvideo contents of encrypted data. Other examples of keys and theencryption and decryption process were described above with respect toFIGS. 2-3.

In some embodiments, retention component 434 can be configured to manageretention and deletion of video content stored in storage device 440. Insome embodiments, retention component 434 can be configured toperiodically, on demand, or requested by storage device 440 to check theretention policy (e.g., retention data 448 and 466) of video content(e.g., encrypted video footage data 441 and encrypted video collectiondata 460) stored on storage device 440 to determine whether to delete orrequest storage device 440 to delete expired video content.

In some embodiments, storage device 440 can correspond to storage device130 of FIG. 1A. Like storage device 130, storage device 440 can beconfigured to provide long term storage of video content. In someembodiments, storage device 440 can receive video footage, videocollections, video clusters, metadata files, and encrypted versionsthereof from video management system 420. Then, storage device 440 maystore the received video data in association with respective retentiondata 448 and 466 controlling how long each video data is to be retainedin storage device 440. In some embodiments, storage device 440 can beconfigured to delete video data, e.g., encrypted video collection data A460, whose retention data 466 indicates that associated video datashould not be retained any longer.

FIG. 5 illustrates a method 500 for limiting video surveillance toauthorized uses, according to some embodiments. In some embodiments, asurveillance system (e.g., video managing system 101 of FIG. 1A) or avideo management system (e.g., video managing system 120) in thesurveillance system can be configured to perform one or more steps ofmethod 500.

In step 502, the surveillance system receives a video footage of an areaunder surveillance from a security camera. In some embodiments, thevideo footage can be received as a video stream or as recorded videofootage. In some embodiments, the video footage can be tagged withmetadata including, for example, one or more location indicators, dataidentifying the security camera, timestamps, etc. In some embodiments,the metadata includes a location indicator that identifies the areaunder surveillance. For example, the location indicator may be a GPScoordinate, a landmark, a street name, an address, etc.

In step 504, the surveillance system analyzes the video footage todetect a plurality of events in the video footage. In some embodiments,the plurality of events can include classes of targets such as persons,vehicles, weapons, etc. In some embodiments the plurality of events caninclude specific targets such as a specific person of interest, aspecific vehicle as identified by a license plate, etc. As describedabove with respect to FIG. 1A, the surveillance system can partition thevideo footage into a plurality of video segments and select one or morerepresentative video frames for each video segment to capture prominentfeatures detected in the video segment. In some embodiments, thesurveillance system can also tag each video segment with any associateddetected targets.

In some embodiments, the surveillance system can generate a metadatafile that includes an index of features to map detected features towhich portions of the video footage in which each feature is detected.Similarly, the surveillance system can generate the metadata file toinclude an index of targets to map detected targets to which portions ofthe video footage in which each target is detected. For example, thesurveillance system may associate each target with one or more videoframes, one or more video segments, one or more video clusters, one ormore time ranges in the video footage, or a combination thereof.

In step 506, the surveillance system generates a video collectioncomprising a copy of a plurality of portions of the video footage basedon applying a watchlist indicating an approved event to the analyzedvideo footage, wherein each copied portion is detected to include theapproved event. In some embodiments, the surveillance system cangenerate the video collection using the metadata file. For example, thesurveillance system may search the index of events in the metadata fileto identify the plurality of portions of the video footage that capturedvideo of the approved event. Then, the surveillance system may copy theplurality of identified portions to the video collection.

In step 508, the surveillance system encrypts the video footage using afirst key to restrict a first user from accessing other portions of thevideo footage that do not include the approved event.

In some embodiments, the first key is not accessible by the first user.In some embodiments, after encrypting the video footage, thesurveillance system can transmit the first key to an authorizationsystem configured to store and secure keys for decrypting encryptedvideo footages. In some embodiments, the surveillance system may receivea notification from the authorization system indicating that the firstkey was received and secured. After receipt of this notification, thesurveillance system can be configured to irrecoverably delete the firstkey from the surveillance system to prevent any user having access tothe surveillance system from recovering the deleted first key. Forexample, the surveillance system may delete the first key from itsmemory, wipe a memory location storing the deleted first key, overwritethe memory location, etc.

In some embodiments, the first key may be part of a key pair used toencrypt and decrypt video data and managed by the authorization system.In some embodiments, the key pair includes a public key for encryptingvideo data and includes a private key for decrypting encrypted videodata. In these embodiments, the first key corresponds to the public keyused by the surveillance system to encrypt the video footage, but whichcannot be used to decrypt the video footage once encrypted. As describedabove, the authorization system can store the private key, which cannotbe accessed by the surveillance system. Accordingly, once thesurveillance system encrypts the video footage using the first key(i.e., the public key), the surveillance system cannot decrypt theencrypted video footage.

In step 510, the surveillance system provides the first user access tothe video collection. In some embodiments, the video collection can bestored in a storage device in an unencrypted format. Therefore, thefirst user having access to the storage device will be able to accessand view the video collection of the approved target indicated in thewatchlist.

In some embodiments, the video collection can be encrypted with a secondkey associated with the first user before being stored in the storagedevice. By doing so, only users such as the first user having access tothe second key will be able to decrypt and access video contents in theencrypted video collection. In some embodiments, the second key may be apublic key in a key pair including the public key and the private key.In these embodiments, only users such as the first user having access tothe private key associated with the second key will be able to decryptthe encrypted video collection.

FIG. 6 illustrates a flowchart of a method for authorizing videosurveillance on previously collected video footage controlling,according to some embodiments. In some embodiments, a surveillancesystem (e.g., video managing system 101 of FIG. 1A) or a videomanagement system (e.g., video managing system 120) in the surveillancesystem can be configured to perform one or more steps of method 600.

In step 602, the surveillance system receives a request/direction froman authorization system to apply an updated watchlist associated with auser to an encrypted video footage. For example, as described above withrespect to FIG. 5, a first watchlist associated with the user may havebeen previously applied to an unencrypted video footage. In thisexample, the updated watchlist may represent a modified version of thefirst watchlist.

In some embodiments, similar to the first watchlist, the updatedwatchlist may include one or more targets and one or more respectivecollection policies. In some embodiments, a collection policy associatedwith a target specifies the parameters under which captured video ofthat target may be accessed. In some embodiments, the updated watchlistincludes a second approved target whose captured video is authorized forviewing by the user. In some embodiments, the updated watchlist includesa disapproved target whose captured video is not authorized for viewingby the user. In these embodiments, the surveillance system can beconfigured to remove portions of video from a first video collectionassociated with the user to comply with the updated watchlist, as willbe further described below.

In step 604, the surveillance system decrypts the encrypted videofootage previously encrypted by a first encryption key by applying adecryption key received from the authorization system. For example, asdescribed above with respect to FIGS. 2-3, the first encryption key andthe decryption key may be a public key and a private key, respectively,of a key pair managed by the authorization system.

In step 606, the surveillance system generates a second video collectionincluding a copy of a second plurality of portions from the decryptedvideo footage based on the updated watchlist, wherein each portioncomprises an approved event in the updated watchlist. In someembodiments, step 606 can include one or more of steps 606A-B.

In step 606A, the surveillance system determines the second plurality ofportions based on a metadata file associated with the video footage. Forexample, the metadata file may have been previously generated when theoriginally received video footage was analyzed, as described above withrespect to FIG. 5. In some embodiments, the metadata file includes anindex that identifies portions of the video footage in which each eventis detected. Accordingly, the surveillance system may search the indexto identify a plurality of portions of the video footage that includeone or more approved events in the updated watchlist. Similarly, thesurveillance system may search the index to identify portions of thevideo that include one or more disapproved events in the updatedwatchlist to prevent these portions from being viewable to the user.

In step 606B, the surveillance system modifies the first videocollection associated with the user to generate the second videocollection. In some embodiments, the surveillance system can copy thesecond plurality of portions to the first video collection. In someembodiments, the surveillance system can redact or discard portions ofthe first video collection that include one or more disapproved eventsindicated in the updated watchlist.

In some embodiments, the surveillance system may have previouslygenerated a metadata file for the first video collection. In theseembodiments, the surveillance system can update the metadata file byupdating a stored index and/or update a change file indicating changesmade between the first video collection and the second video collection.

In step 608, the surveillance system encrypts the video footage using asecond encryption key to restrict the user from accessing other portionsof the video footage not approved by the updated watchlist. In someembodiments, the surveillance system encrypts the video footage using asecond encryption key different than the first encryption key of step604.

In some embodiments, the surveillance system can generate the secondencryption key. In these embodiments, the surveillance system cantransmit the second encryption key to the authorization system.Subsequently, in response to receiving a notification from theauthorization system indicating that the second encryption key wasreceived and secured, the surveillance system can irrecoverably deletethe second encryption key.

In some embodiments, the surveillance system receives the secondencryption key with the decryption key from the authorization system. Inthese embodiments, the second encryption key may be a public key of asecond key pair including the public key and a private key that candecrypt data encrypted by the public key. In some embodiments, theauthorization system can be configured to manage the second key pair andthe surveillance system has no access to the private key of the secondkey pair. Accordingly, once the surveillance system encrypts the videofootage using the second encryption key, the surveillance system willnot be able to decrypt the encrypted video footage and thereby preventsthe user from accessing video content from the encrypted video footage.

In step 610, the surveillance system provides the user access to thesecond video collection. As described above, the second video collectionmay be encrypted by a crypto key associated with the user to preventother users from accessing video content stored in the second videocollection. Additionally, the updated metadata file associated with thesecond video collection may also be encrypted by the crypto key.

FIG. 7 illustrates an example of a computing device 700, according tosome embodiments. Device 700 can be a host computing device connected toa network. For example, device 700 may be an example implementation ofone or more of the devices or systems described above with respect toFIGS. 1A-B and 4. Device 700 can be a client computer or a server. Asshown in FIG. 7, device 700 can be any suitable type ofmicroprocessor-based device, such as a personal computer, work station,or server. The device can include, for example, one or more of processor710, input device 720, output device 730, storage 740, and communicationdevice 760. Input device 720 and output device 730 can generallycorrespond to those described above and can either be connectable orintegrated with the computing device.

Input device 720 can be any suitable device that provides input, such asa touchscreen, keyboard or keypad, mouse, or voice-recognition device.Output device 730 can be any suitable device that provides output, suchas a touchscreen, haptics device, or speaker.

Storage 740 can be any suitable device that provides storage, such as anelectrical, magnetic, or optical memory including a RAM, cache, harddrive, or removable storage disk. Communication device 760 can includeany suitable device capable of transmitting and receiving signals over anetwork, such as a network interface chip or device. The components ofthe computing device can be connected in any suitable manner, such asvia a physical bus, or wirelessly.

Software 750, which can be stored in storage 740 and executed byprocessor 710, can include, for example, the programming that embodiesthe functionality of the present disclosure (e.g., as embodied in thedevices described above). For example, software 750 may include systemsoftware (e.g., an operating system), application software, or securitysoftware.

Software 750 can also be stored and/or transported within anynon-transitory, computer-readable storage medium for use by or inconnection with an instruction execution system, apparatus, or device,such as those described above, that can fetch instructions associatedwith the software from the instruction execution system, apparatus, ordevice and execute the instructions. In the context of this disclosure,a computer-readable storage medium can be any medium, such as storage740, that can contain or store programming for use by or in connectionwith an instruction-execution system, apparatus, or device.

Software 750 can also be propagated within any transport medium for useby or in connection with an instruction-execution system, apparatus, ordevice, such as those described above, that can fetch instructionsassociated with the software from the instruction-execution system,apparatus, or device and execute the instructions. In the context ofthis disclosure, a transport medium can be any medium that cancommunicate, propagate, or transport programming for use by or inconnection with an instruction-execution system, apparatus, or device.The transport readable medium can include, but is not limited to, anelectronic, magnetic, optical, electromagnetic, or infrared wired orwireless propagation medium.

Device 700 may be connected to a network, which can be any suitable typeof interconnected communication system. The network can implement anysuitable communications protocol and can be secured by any suitablesecurity protocol. The network can comprise network links of anysuitable arrangement that can implement the transmission and receptionof network signals, such as wireless network connections, T1 or T3lines, cable networks, DSL, or telephone lines.

Device 700 can implement any operating system suitable for operating onthe network. Software 750 can be written in any suitable programminglanguage, such as C, C++, Java, or Python. In various embodiments,application software embodying the functionality of the presentdisclosure can be deployed in different configurations, such as in aclient/server arrangement, for example.

The foregoing description, for purpose of explanation, has madereference to specific embodiments. However, the illustrative discussionsabove are not intended to be exhaustive or to limit the disclosure tothe precise forms disclosed. Many modifications and variations arepossible in view of the above teachings. The embodiments were chosen anddescribed in order to best explain the principles of the techniques andtheir practical applications. Others skilled in the art are therebyenabled to best utilize the techniques and various embodiments, withvarious modifications, that are suited to the particular usecontemplated.

Although the disclosure and examples have been fully described withreference to the accompanying figures, it is to be noted that variouschanges and modifications will be apparent to those skilled in the art.Such changes and modifications are to be understood as being includedwithin the scope of the disclosure and examples as defined by theclaims.

What is claimed is:
 1. A method for limiting video surveillance toauthorized uses, comprising: at a surveillance system: receiving a videofootage of an area under surveillance from a security camera; analyzingthe video footage to detect a plurality of events in the video footage;generating a video collection comprising a copy of a plurality ofportions of the video footage based on applying a watchlist indicatingan approved event of interest to the analyzed video footage, whereineach copied portion is detected to include the approved event;encrypting the video footage using a first key to restrict a first userfrom accessing other portions of the video footage that do not includethe approved event; and providing the first user access to the videocollection.
 2. The method of claim 1, wherein the approved eventcomprises a class of targets or a specific target.
 3. The method ofclaim 1, wherein the watchlist comprises an indication of the approvedevent and a plurality of associated parameters for video surveillance,and the method comprises: determining which video segments of the videofootage meets the plurality of parameters and includes the approvedevent, wherein the plurality of copied portions of the video footagecomprises the determined video segments.
 4. The method of claim 3,wherein the plurality of parameters comprises one or more locations andone or more time periods approved for surveilling the target.
 5. Themethod of claim 3, wherein the plurality of parameters includes alocation approved for surveilling the event, and the method comprises:determining that the video footage is associated with the approvedlocation before generating the video collection.
 6. The method of claim5, wherein determining that the video footage is associated with theapproved location comprises one of: analyzing the video footage toidentify one or more features that uniquely identifies the location; ordetermining that metadata of the received video footage indicates thelocation.
 7. The method of claim 1, comprising: transmitting the firstkey to an authorization system that controls access to a plurality ofencrypted video footages; receiving a notification from theauthorization system indicating that the first key was received andsecurely stored by the authorization system; and irrecoverably deletethe first key from the surveillance system.
 8. The method of claim 1,wherein the first key is a first public key, comprising: receiving thefirst public key from an authorization system that controls access to aplurality of encrypted video footage, wherein the authorization systemstores a first key pair comprising the first public key and a firstprivate key, and wherein the encrypted video footage can only bedecrypted using the first private key located at the authorizationsystem.
 9. The method of claim 1, wherein providing the users access tothe video collection comprises: encrypting the video collection using asecond key pair associated with the user to prevent other users frombeing able to access the video collection, wherein the video collectionis decrypted using the second key pair when access to video content inthe video collection is requested by the user.
 10. The method of claim1, wherein analyzing the video footage comprises: generating a metadatafile comprising an index that maps each event from the plurality oftargets to one or more portions of the video footage detected to includethe event.
 11. The method of claim 10, comprising: encrypting themetadata file with the first key.
 12. The method of claim 10, whereingenerating the video collection comprises: identifying the plurality ofportions of the video footage that include the approved event bysearching the metadata file; and copying the plurality of identifiedportions to the video collection.
 13. The method of claim 10,comprising: generating a second metadata file that includes the indexfor the approved event and that removes or redacts the index tonon-approved events of the plurality of events.
 14. The method of claim1, wherein the approved event is a first approved event, wherein thewatchlist indicates a second disapproved event whose captured videocannot be accessed by the users, and wherein each copied portion isdetected to include the first approved event and not include the seconddisapproved event.
 15. The method of claim 1, comprising: receiving arequest from an authorization system to apply an updated watchlistassociated with the users to the encrypted video footage; decrypting theencrypted video footage encrypted by a first encryption key using adecryption key received from the authorization system; generate a secondvideo collection including a copy of a second plurality of portions fromthe decrypted video footage based on the updated watchlist, wherein eachportion comprises one or more approved events from the updatedwatchlist; encrypting the decrypted video footage using a secondencryption key to restrict the users from accessing other portions ofthe video footage not approved by the updated watchlist; and provide theusers access to the second video collection.
 16. A surveillance systemfor limiting video surveillance to authorized uses, comprising: one ormore processors; memory comprising a local storage; and one or moreprograms, wherein the one or more programs are stored in the memory andconfigured to be executed by the one or more processors, the one or moreprograms including instructions for: receiving a video footage of anarea under surveillance from a security camera; analyzing the videofootage to detect a plurality of events in the video footage; generatinga video collection comprising a copy of a plurality of portions of thevideo footage based on applying a watchlist indicating an approved eventof interest to the analyzed video footage, wherein each copied portionis detected to include the approved event; encrypting the video footageusing a first key to restrict a first user from accessing other portionsof the video footage that do not include the approved event; andproviding the first user access to the video collection.
 17. The systemof claim 16, wherein the approved event comprises a class of targets ora specific target.
 18. The system of claim 16, wherein the watchlistcomprises an indication of the approved event and a plurality ofassociated parameters for video surveillance, and the program includesinstructions for: determining which video segments of the video footagemeets the plurality of parameters and includes the approved event,wherein the plurality of copied portions of the video footage comprisesthe determined video segments.
 19. The system of claim 18, wherein theplurality of parameters comprises one or more locations and one or moretime periods approved for surveilling the target.
 20. The system ofclaim 18, wherein the plurality of parameters includes a locationapproved for surveilling the event, and the method comprises:determining that the video footage is associated with the approvedlocation before generating the video collection.
 21. The system of claim20, wherein determining that the video footage is associated with theapproved location comprises one of: analyzing the video footage toidentify one or more features that uniquely identifies the location; ordetermining that metadata of the received video footage indicates thelocation.
 22. The system of claim 16, wherein the program includesinstructions for: transmitting the first key to an authorization systemthat controls access to a plurality of encrypted video footages;receiving a notification from the authorization system indicating thatthe first key was received and securely stored by the authorizationsystem; and irrecoverably delete the first key from the surveillancesystem.
 23. The system of claim 16, wherein the first key is a firstpublic key, comprising: receiving the first public key from anauthorization system that controls access to a plurality of encryptedvideo footage, wherein the authorization system stores a first key paircomprising the first public key and a first private key, and wherein theencrypted video footage can only be decrypted using the first privatekey located at the authorization system.
 24. The system of claim 16,wherein providing the users access to the video collection comprises:encrypting the video collection using a second key pair associated withthe user to prevent other users from being able to access the videocollection, wherein the video collection is decrypted using the secondkey pair when access to video content in the video collection isrequested by the user.
 25. The system of claim 16, wherein analyzing thevideo footage comprises: generating a metadata file comprising an indexthat maps each event from the plurality of targets to one or moreportions of the video footage detected to include the event.
 26. Thesystem of claim 25, comprising: encrypting the metadata file with thefirst key.
 27. The system of claim 25, wherein generating the videocollection comprises: identifying the plurality of portions of the videofootage that include the approved event by searching the metadata file;and copying the plurality of identified portions to the videocollection.
 28. The system of claim 25, wherein the program includesinstructions for: generating a second metadata file that includes theindex for the approved event and that removes or redacts the index tonon-approved events of the plurality of events.
 29. The system of claim16, wherein the approved event is a first approved event, wherein thewatchlist indicates a second disapproved event whose captured videocannot be accessed by the users, and wherein each copied portion isdetected to include the first approved event and not include the seconddisapproved event.
 30. The system of claim 16, wherein the programincludes instructions for: receiving a request from an authorizationsystem to apply an updated watchlist associated with the users to theencrypted video footage; decrypting the encrypted video footageencrypted by a first encryption key using a decryption key received fromthe authorization system; generate a second video collection including acopy of a second plurality of portions from the decrypted video footagebased on the updated watchlist, wherein each portion comprises one ormore approved events from the updated watchlist; encrypting thedecrypted video footage using a second encryption key to restrict theusers from accessing other portions of the video footage not approved bythe updated watchlist; and provide the users access to the second videocollection.
 31. A non-transitory computer-readable storage mediumcomprising one or more programs for targeted video surveillanceprocessing, wherein the one or more programs, when executed by one ormore processors, cause the one or more processors to: receiving a videofootage of an area under surveillance from a security camera; analyzingthe video footage to detect a plurality of events in the video footage;generating a video collection comprising a copy of a plurality ofportions of the video footage based on applying a watchlist indicatingan approved event of interest to the analyzed video footage, whereineach copied portion is detected to include the approved event;encrypting the video footage using a first key to restrict a first userfrom accessing other portions of the video footage that do not includethe approved event; and providing the first user access to the videocollection.
 32. The non-transitory computer-readable storage medium ofclaim 31, wherein the approved event comprises a class of targets or aspecific target.
 33. The non-transitory computer-readable storage mediumof claim 31, wherein the watchlist comprises an indication of theapproved event and a plurality of associated parameters for videosurveillance, and the processor is caused to: determine which videosegments of the video footage meets the plurality of parameters andincludes the approved event, wherein the plurality of copied portions ofthe video footage comprises the determined video segments.
 34. Thenon-transitory computer-readable storage medium of claim 33, wherein theplurality of parameters comprises one or more locations and one or moretime periods approved for surveilling the target.
 35. The non-transitorycomputer-readable storage medium of claim 33, wherein the plurality ofparameters includes a location approved for surveilling the event, andthe method comprises: determining that the video footage is associatedwith the approved location before generating the video collection. 36.The non-transitory computer-readable storage medium of claim 35, whereindetermining that the video footage is associated with the approvedlocation comprises one of: analyzing the video footage to identify oneor more features that uniquely identifies the location; or determiningthat metadata of the received video footage indicates the location. 37.The non-transitory computer-readable storage medium of claim 16, whereinthe processor is caused to: transmit the first key to an authorizationsystem that controls access to a plurality of encrypted video footages;receive a notification from the authorization system indicating that thefirst key was received and securely stored by the authorization system;and irrecoverably delete the first key from the surveillance system. 38.The non-transitory computer-readable storage medium of claim 31, whereinthe first key is a first public key, comprising: receiving the firstpublic key from an authorization system that controls access to aplurality of encrypted video footage, wherein the authorization systemstores a first key pair comprising the first public key and a firstprivate key, and wherein the encrypted video footage can only bedecrypted using the first private key located at the authorizationsystem.
 39. The non-transitory computer-readable storage medium of claim31, wherein providing the users access to the video collectioncomprises: encrypting the video collection using a second key pairassociated with the user to prevent other users from being able toaccess the video collection, wherein the video collection is decryptedusing the second key pair when access to video content in the videocollection is requested by the user.
 40. The non-transitorycomputer-readable storage medium of claim 31, wherein analyzing thevideo footage comprises: generating a metadata file comprising an indexthat maps each event from the plurality of targets to one or moreportions of the video footage detected to include the event.
 41. Thenon-transitory computer-readable storage medium of claim 40, comprising:encrypting the metadata file with the first key.
 42. The non-transitorycomputer-readable storage medium of claim 40, wherein generating thevideo collection comprises: identifying the plurality of portions of thevideo footage that include the approved event by searching the metadatafile; and copying the plurality of identified portions to the videocollection.
 43. The non-transitory computer-readable storage medium ofclaim 40, wherein the program includes instructions for: generating asecond metadata file that includes the index for the approved event andthat removes or redacts the index to non-approved events of theplurality of events.
 44. The non-transitory computer-readable storagemedium of claim 31, wherein the approved event is a first approvedevent, wherein the watchlist indicates a second disapproved event whosecaptured video cannot be accessed by the users, and wherein each copiedportion is detected to include the first approved event and not includethe second disapproved event.
 45. The non-transitory computer-readablestorage medium of claim 31, wherein the processor is caused to: receivea request from an authorization system to apply an updated watchlistassociated with the users to the encrypted video footage; decrypt theencrypted video footage encrypted by a first encryption key using adecryption key received from the authorization system; generate a secondvideo collection including a copy of a second plurality of portions fromthe decrypted video footage based on the updated watchlist, wherein eachportion comprises one or more approved events from the updatedwatchlist; encrypt the decrypted video footage using a second encryptionkey to restrict the users from accessing other portions of the videofootage not approved by the updated watchlist; and provide the usersaccess to the second video collection.